VisualApplets
Tutorial and Examples

Basic Operators

We start with the basic operators in our design as shown in the following figure. We can use all the default parameters and link properties.

Synchronization

Next we need to synchronize the data streams of both cameras. This can simply be done by using operator SYNC. The SYNC operator controls it's inputs so that the outputs are fully synchronized. As you can see, we used two buffer modules before the synchronization. We need to use them because we cannot ensure that both cameras are 100% synchronous.

Although we synchronized our image data, we still need pay attention to some constraints:

Overlay-Blend

The last step to do is to implement the overlay blend to combine both images. For convenience, we use a hierarchical box for the overlay-blend implementation.

The overlay blend has to be individually applied to all color components. Thus we need to split our links into all three color components. The overlay blends are placed in hierarchical boxes once again. Thus we have three hierarchical boxes inside the current hierarchical box. You can use copy and paste to duplicate the boxes for each color component.

We will have varying possibilities to realize the equation given above using VisualApplets operators. In general, you should always try to adapt and optimize an equation for FPGA use before implementing it.

The equation includes some multiplications and divisions with constant values. These operations are available in VisualApplets but cannot be very efficiently implemented on FPGAs. If the multiplication or division can be written as a power of two multiplication we can use a simple bit shift for implementation. A bit shift will require no resources at all! We will simplify

to

which is a simple right shift by 7 bit i.e. we discard the seven least significant bit.

Next, there are some parts in the equation like 255 minus ColorA. This is a simple color inversion. White becomes dark, dark becomes white. An inversion of an value can also be made using a simple inversion of each bit. This allows us to finally rewrite the equation:

Equation 2. Optimized Overlay Blend for FPGA Implementation

We can now start our implementation. The remaining operations in the equation are available as VisualApplets operators. We will need

Verification using the Simulation

1. We can easily verify the functionality of our overflow-blend using the VisualApplets simulation. Use two simulation sources to inject an image for both cameras. Place simulation probes to any link you like. The simulation results will show you the overlay of both images. If interested, you can also check the intermediate results of the overlay in the hierarchical boxes.

   
2. The verification of the synchronization is not possible with the simulation as the simulation will only reflect the functional parts of the applets and not the timing. A verification in hardware is required.

Adding a Trigger

As mentioned, both cameras have to run at the same frame rate to avoid an overflow on one of the buffers. This can be easily solved using a trigger system.

Basic Operators

We will need the standard design for two cameras using two cameras modules, two buffers and one DMA.

Multiplex Images

To multiplex the images, we use operator InsertImage. This operator will forward the images of it's inputs one after another. For example, if we use two inputs, the operator will forward the images of input 0, 1, 0, 1, 0, 1, ... Thus, after one input has been processed, the next one will be used. Therefore, it is required that the number of frames at both inputs is the same i.e. the inputs will have the same frame rate.

As you can see, we placed the image buffer modules before the insertion. This is because the InsertImage operator has no buffer itself. While images of input 0 are forwarded, input one is blocked, thus camera images need to be buffered to avoid the loss of data.

For each input (I0, I1, ..) the operator provides a second input called Ins0, Ins1, ... This input is a control input to control whether images shall be used or discarded. Thus an input can be skipped if value 0 is at the Ins input and will be used if value 1 is provided. The input groups I and Ins have to be synchronous, i.e. they have to be sourced by the same M-type operator through an arbitrary network of O-type operators. See 'Operator Types' for more information. As both inputs are synchronous, they will need to have the same image dimension. InsertImage will use the first pixel on the Ins inputs to decide whether an image is used or discarded. As you can see in the previous screenshots, we simply used a CONST operator to enable the use of the images. Set the output link bit depth of the CONST operator to one bit and the parameter "Value" to 1 to enable the insertion.

Please note: If the value at one of the Ins inputs is zero, the operator will still need to process the images on this inputs. A value zero does not mean, that no image has to be provided. If no image is provided at an input, the operator waits until an image is available.

As mentioned always one of the inputs is opened at a time, while the others are blocked. The images have to be sourced asynchronously e.g. from different buffers. In the following figure, you can see two configurations which will not work. You will not get any results from the applet as it is blocking itself. It is a so called "deadlock" condition.

Figure 215. Deadlock Configurations using InsertImage

Always check if one of the inputs can be buffered when stopped, while the other one is active. Do not forget that deadlock situations cannot be detected by the DRC nor the simulation. More information on deadlocks can be found in 'Timing Synchronization'.